ABSTRACT
Background: The COVID-19 pandemic has highlighted the importance of mental wellbeing. The identification and implementation of quality measures can improve health outcomes and patient experience. The objective was to identify and define a core set of valid and relevant pediatric mental health quality measures that will support health system evaluation and quality improvement in British Columbia, Canada. Methods: The study consisted of four phases. First, a comprehensive database search identified valid pediatric quality measures focused on mental health and substance use (MH/SU). Second, the identified quality measures were mapped to focus areas, which were then prioritized by two stakeholder groups consisting of 26 members. Third, up to two representative measures for each prioritized focus area were pre-selected by an expert panel (n = 9). And fourth, a three-step modified Delphi approach was employed to (1) assess each quality measure on a 7-point Likert scale against three relevance criteria (representative of a quality problem, value to intended audience and actionable), (2) discuss the results, and (3) select and rank the most relevant measures. Forty-eight stakeholders were invited to participate; of those 24 completed the round 1 survey, 21 participated in the round 2 discussion and 18 voted in the round 3 selection and ranking survey. For round 1, consensus was determined when at least 70% of the response rates were within the range of five to seven. For round 3, Kendall's coefficient of concordance W was used as an estimator of inter-rater reliability. Results: One-hundred pediatric mental health quality measures were identified in the database search. Of those, 37 were mapped to ten focus areas. Pre-selection resulted in 19 representative measures moving forward to the Delphi study. Eleven measures met the consensus thresholds and were brought forward to the round 2 discussion. Round 3 ranking showed moderate to strong raters' agreement (Kendall's W = 0.595; p < 0.01) and resulted in the following five highest-ranked measures: level of satisfaction after discharge from inpatient admission due to MH/SU, number of patients experiencing seclusion or restraint, length of time from eating disorder referral to assessment, number of ED visits due to MH/SU, and number of readmissions to ED. Conclusion: The selected core set of valid and relevant pediatric quality measures will support sustainable system change in British Columbia. The five top-ranked measures will be refined and tested for data collection feasibility before being implemented in the province.
ABSTRACT
Background The COVID-19 pandemic has highlighted the importance of mental wellbeing. The identification and implementation of quality measures can improve health outcomes and patient experience. The objective was to identify and define a core set of valid and relevant pediatric mental health quality measures that will support health system evaluation and quality improvement in British Columbia, Canada. Methods The study consisted of four phases. First, a comprehensive database search identified valid pediatric quality measures focused on mental health and substance use (MH/SU). Second, the identified quality measures were mapped to focus areas, which were then prioritized by two stakeholder groups consisting of 26 members. Third, up to two representative measures for each prioritized focus area were pre-selected by an expert panel (n = 9). And fourth, a three-step modified Delphi approach was employed to (1) assess each quality measure on a 7-point Likert scale against three relevance criteria (representative of a quality problem, value to intended audience and actionable), (2) discuss the results, and (3) select and rank the most relevant measures. Forty-eight stakeholders were invited to participate;of those 24 completed the round 1 survey, 21 participated in the round 2 discussion and 18 voted in the round 3 selection and ranking survey. For round 1, consensus was determined when at least 70% of the response rates were within the range of five to seven. For round 3, Kendall's coefficient of concordance W was used as an estimator of inter-rater reliability. Results One-hundred pediatric mental health quality measures were identified in the database search. Of those, 37 were mapped to ten focus areas. Pre-selection resulted in 19 representative measures moving forward to the Delphi study. Eleven measures met the consensus thresholds and were brought forward to the round 2 discussion. Round 3 ranking showed moderate to strong raters' agreement (Kendall's W = 0.595;p < 0.01) and resulted in the following five highest-ranked measures: level of satisfaction after discharge from inpatient admission due to MH/SU, number of patients experiencing seclusion or restraint, length of time from eating disorder referral to assessment, number of ED visits due to MH/SU, and number of readmissions to ED. Conclusion The selected core set of valid and relevant pediatric quality measures will support sustainable system change in British Columbia. The five top-ranked measures will be refined and tested for data collection feasibility before being implemented in the province.
ABSTRACT
Introduction: Adoption of virtual health (VH) solutions in healthcare has been challenging; this changed rapidly after implementation of physical distancing measures due to the COVID-19 pandemic. In response to the pandemic, British Columbia's Children's and Women's sub-specialty hospitals rapidly trained and scaled up support to equip staff and clinicians to use VH. Methods: Ninety-minute live online training workshops and frequently updated online support materials were offered for 6 weeks. Training was monitored via feedback collected at training sessions and a brief post-training survey. After training completion, a second survey was circulated to measure utilization outcomes and experiences with VH. Results: Eight hundred and ninety-five participants representing 82% of staff requiring support were trained through 101 sessions; 348 (38.9%) and 272 (30.4%) responses were collected for the monitoring and outcome surveys, respectively. Overall, 89% agreed that training was relevant to their needs; participants indicated average 58.1% (SD = 26.6) and 60.6% (SD = 25.2) increase in knowledge and confidence in VH after training; 90.1% had booked or conducted VH sessions. Increase in confidence was more pronounced in participants with lesser previous exposure to VH, but number of sessions conducted post-training and percentage of successful sessions were independent of previous exposure. For future training and support, participants suggested subject-tailored trainings, asynchronous trainings, and availability of experienced users. Discussion: Training is key to success of VH implementation. Moving forward, core competencies in VH should be developed to support standardization and allow for evaluation and quality improvement. Incorporation of VH training in continuous professional development and onboarding is also highly recommended.